An AI that can produce photorealistic videos of sign language interpreters from speech could improve accessibility by removing the need for humans. Ben Saunders at the University of Surrey, UK, and his colleagues used a neural network that converts spoken language into sign language. The system, called SignGAN, then maps these signs on to a 3D model of the human skeleton.

Quickly converting speech to text allows deaf and hard of hearing people to interactively follow along with live speech. Doing so reliably requires a combination of perception, understanding, and speed that neither humans nor machines possess alone. In this article, we discuss how our Scribe system combines human labor and machine intelligence in real time to reliably convert speech to text with less than 4s latency. To achieve this speed while maintaining high accuracy, Scribe integrates automated assistance in two ways. First, its user interface directs workers to different portions of the audio stream, slows down the portion they are asked to type, and adaptively determines segment length based on typing speed.